[Improve][Transform-V2] Improve sql transform exception to locate error expression #9227

misi1987107 · 2025-04-25T01:54:10Z

Purpose of this pull request

resolve issue #9125

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
If you are contributing the connector code, please check that the following files are updated:
1. Update plugin-mapping.properties and add new connector information in it
2. Update the pom file of seatunnel-dist
3. Add ci label in label-scope-conf
4. Add e2e testcase in seatunnel-e2e
5. Update connector plugin_config
Update the release-note.

Copilot

Pull Request Overview

This pull request addresses issue #9125 by enhancing error handling in the SQL transformation logic. Key changes include:

Wrapping the SQL expression evaluation in a try-catch block to throw a more specific exception.
Introducing new exception classes and error codes to better pinpoint errors in SQL processing.
Adding new files for SqlTransformException and SqlTransformErrorCode to support the improved exception handling.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLEngine.java	Added exception handling for SQL expression evaluation and improved error reporting.
seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/exception/SqlTransformException.java	New exception class for SQL transformation errors.
seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/exception/SqlTransformErrorCode.java	New error code enum defining the SQL transformation error codes.

Comments suppressed due to low confidence (1)

seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLEngine.java:295

Consider adding a space after the comma in the error message for improved readability, e.g., "Failed to execute sql expression, the error expression is %s".

String.format("Failed to execute sql expression,the error expression is %s", expression),

Hisoka-X

Please update unit test.

Hisoka-X · 2025-05-15T11:15:11Z

...nsforms-v2/src/main/java/org/apache/seatunnel/transform/exception/SqlTransformErrorCode.java

+
+import org.apache.seatunnel.common.exception.SeaTunnelErrorCode;
+
+public enum SqlTransformErrorCode implements SeaTunnelErrorCode {


Please reuse

seatunnel/seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/exception/TransformCommonError.java

Line 33 in 3783816

public class TransformCommonError {

…ansforCommonError

Hisoka-X · 2025-05-16T02:56:31Z

...unnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLEngine.java

+                    if (e instanceof TransformException) {
+                        throw e;


why we should special care TransformException?

I previously thought that the computeForValue() method threw a TransformException(), so I should throw it outward.But I found this exception can be merged in the upper layer.

Hisoka-X · 2025-05-16T03:06:14Z

...ansforms-v2/src/main/java/org/apache/seatunnel/transform/exception/TransformCommonError.java

@@ -67,4 +68,11 @@ public static TransformException cannotFindInputTableError(String transform, Str
        params.put("transform", transform);
        return new TransformException(INPUT_TABLE_NOT_FOUND, params);
    }
+
+    public static TransformException sqlTransformError(String transform, String expression) {


Suggested change

public static TransformException sqlTransformError(String transform, String expression) {

public static TransformException sqlExpressionError(String transform, String expression) {

Hisoka-X · 2025-05-16T03:07:01Z

...orms-v2/src/main/java/org/apache/seatunnel/transform/exception/TransformCommonErrorCode.java

+            "The input table '<table>' of '<transform>' transform not found in upstream schema"),
+    EXPRESSION_EXECUTE_ERROR(
+            "TRANSFORM_COMMON-06",
+            "The expression '<expression>' of '<transform>' transform execute failed");


Suggested change

"The expression '<expression>' of '<transform>' transform execute failed");

"The expression '<expression>' of SQL transform execute failed");

The transform should always be SQL.

Hisoka-X · 2025-05-16T03:08:45Z

seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/sql/SQLTransformTest.java

+                        + "CAST(`FIELD1` AS STRING) AS FIELD1, "
+                        + "CAST(`FIELD1` AS decimal(22,4)) AS FIELD2, "
+                        + "CAST(`FIELD3` AS decimal(22,0)) AS FIELD3 "
+                        + "from dual";


I think we should handle where statement too.

2.remove throw TransformException 3.rename sqlTransformError to sqlExpressionError 4.modify EXPRESSION_EXECUTE_ERROR description 5.add where statement processing

Hisoka-X · 2025-05-16T10:46:19Z

...unnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLEngine.java

+                return null;
+            }
+        } catch (Exception e) {
+            throw TransformCommonError.sqlWhereStatementError(selectBody.getWhere().toString());


We need add the exception into cause stack. Please refer

seatunnel/seatunnel-common/src/main/java/org/apache/seatunnel/common/exception/CommonError.java

Line 66 in 454a88f

String identifier, String operation, String fileName, Throwable cause) {

Hisoka-X · 2025-05-20T02:16:58Z

seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/sql/SQLTransformTest.java

@@ -411,7 +411,7 @@ public void tesCastBooleanClausesWithField() {
                                new SeaTunnelRow(new Object[] {Integer.valueOf(1), 0, "false333"}));
                    } catch (Exception e) {
                        Assertions.assertEquals(
-                                "ErrorCode:[COMMON-05], ErrorDescription:[Unsupported operation] - Unsupported CAST AS Boolean: false333",


The new exception looks like lose expression value in message. We should fix it.

already complete

Copilot

Pull Request Overview

This PR enhances SQL transform error reporting by adding specific exception wrappers for expression and WHERE failures, introduces new error codes, and updates tests to validate these detailed messages.

Wraps filter and projection logic in ZetaSQLEngine with try/catch to provide TRANSFORM_COMMON-06 and TRANSFORM_COMMON-07 errors.
Adds new error codes and constructors in TransformCommonErrorCode and TransformException, plus helper methods in TransformCommonError.
Updates existing tests and adds a new test in SQLTransformTest to assert on the improved error messages.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLEngine.java	Wraps filter/project calls to rethrow with context-specific errors
seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/exception/TransformException.java	Adds constructor to preserve original cause
seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/exception/TransformCommonErrorCode.java	Defines new error codes for expression and WHERE errors
seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/exception/TransformCommonError.java	Implements builder methods for the new error codes
seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/sql/SQLTransformTest.java	Updates expected error messages and adds new error-case tests

Comments suppressed due to low confidence (1)

seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/sql/SQLTransformTest.java:473

[nitpick] Consider making this assertion less brittle by matching on the error code and checking that the expression substring appears in the message, rather than comparing the entire error message exactly. This will reduce test failures due to minor formatting changes.

Assertions.assertEquals(...)

Copilot · 2025-05-20T07:38:16Z

seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/sql/SQLTransformTest.java

@@ -397,7 +397,7 @@ public void tesCastBooleanClausesWithField() {
                                new SeaTunnelRow(new Object[] {Integer.valueOf(1), 3, "false"}));
                    } catch (Exception e) {
                        Assertions.assertEquals(
-                                "ErrorCode:[COMMON-05], ErrorDescription:[Unsupported operation] - Unsupported CAST AS Boolean: 3",
+                                "ErrorCode:[TRANSFORM_COMMON-06], ErrorDescription:[The expression 'cast(`int` AS boolean)' of SQL transform execute failed]",


The expected error message uses lowercase cast, but the actual exception uses CAST (uppercase) from the expression's toString(). Update this assertion to match CAST for consistency.

Suggested change

"ErrorCode:[TRANSFORM_COMMON-06], ErrorDescription:[The expression 'cast(`int` AS boolean)' of SQL transform execute failed]",

"ErrorCode:[TRANSFORM_COMMON-06], ErrorDescription:[The expression 'CAST(`int` AS boolean)' of SQL transform execute failed]",

Copilot · 2025-05-20T07:38:17Z

seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/sql/SQLTransformTest.java

@@ -411,7 +411,7 @@ public void tesCastBooleanClausesWithField() {
                                new SeaTunnelRow(new Object[] {Integer.valueOf(1), 0, "false333"}));
                    } catch (Exception e) {
                        Assertions.assertEquals(
-                                "ErrorCode:[COMMON-05], ErrorDescription:[Unsupported operation] - Unsupported CAST AS Boolean: false333",
+                                "ErrorCode:[TRANSFORM_COMMON-06], ErrorDescription:[The expression 'cast(`string` AS boolean)' of SQL transform execute failed]",


The expected error message uses lowercase cast, but the actual exception uses CAST. Update this assertion to expect CAST(string AS boolean).

Suggested change

"ErrorCode:[TRANSFORM_COMMON-06], ErrorDescription:[The expression 'cast(`string` AS boolean)' of SQL transform execute failed]",

"ErrorCode:[TRANSFORM_COMMON-06], ErrorDescription:[The expression 'CAST(`string` AS boolean)' of SQL transform execute failed]",

Copilot · 2025-05-20T07:38:17Z

seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/sql/SQLTransformTest.java

+                                new SeaTunnelRow(new Object[] {1, 123.123, "true"}));
+                    } catch (Exception e) {
+                        Assertions.assertEquals(
+                                "ErrorCode:[TRANSFORM_COMMON-06], ErrorDescription:[The expression 'CAST(`FIELD3` AS decimal (22, 0))' of SQL transform execute failed]",


The spacing in decimal (22, 0) may not match the expression's toString() output (likely decimal(22,0)). Adjust the expected string to CAST(FIELD3 AS decimal(22,0)) to align with the actual error message.

Suggested change

"ErrorCode:[TRANSFORM_COMMON-06], ErrorDescription:[The expression 'CAST(`FIELD3` AS decimal (22, 0))' of SQL transform execute failed]",

"ErrorCode:[TRANSFORM_COMMON-06], ErrorDescription:[The expression 'CAST(`FIELD3` AS decimal(22,0))' of SQL transform execute failed]",

…ansforCommonError

2.remove throw TransformException 3.rename sqlTransformError to sqlExpressionError 4.modify EXPRESSION_EXECUTE_ERROR description 5.add where statement processing

[feature] throw sql transform exception,locate error expression

75e8310

github-actions bot added the Transform-v2 label Apr 25, 2025

[improve] throw TransformException

38eef59

hailin0 requested a review from Copilot April 27, 2025 01:44

Copilot AI reviewed Apr 27, 2025

View reviewed changes

Hisoka-X reviewed Apr 28, 2025

View reviewed changes

misi added 3 commits April 28, 2025 17:49

[improve] add test case

e461da3

[improve] improve code

59d4fda

Merge branch 'dev' into bugfix-9125

f786dac

Hisoka-X reviewed May 15, 2025

View reviewed changes

[improve] remove SqlTransformErrorCode\SqlTrasformException, reuse Tr…

be184d1

…ansforCommonError

Hisoka-X reviewed May 16, 2025

View reviewed changes

[improve] 1.add where statement test case

d76b087

2.remove throw TransformException 3.rename sqlTransformError to sqlExpressionError 4.modify EXPRESSION_EXECUTE_ERROR description 5.add where statement processing

Hisoka-X reviewed May 16, 2025

View reviewed changes

misi added 2 commits May 19, 2025 09:31

[improve] add the exception into cause stack

683530c

Merge remote-tracking branch 'origin/dev' into bugfix-9125

c433c94

Hisoka-X previously approved these changes May 19, 2025

View reviewed changes

github-actions bot added approved reviewed labels May 19, 2025

Hisoka-X changed the title ~~[feature] throw sql transform exception,locate error expression~~ [Improve][Transform-V2] Improve sql transform exception to locate error expression May 19, 2025

[improve] modify throw the sqlTransformException

aeeddb7

misi1987107 dismissed Hisoka-X’s stale review via aeeddb7 May 20, 2025 01:22

github-actions bot removed approved reviewed labels May 20, 2025

Hisoka-X reviewed May 20, 2025

View reviewed changes

nielifeng requested a review from Copilot May 20, 2025 07:36

Copilot AI reviewed May 20, 2025

View reviewed changes

[improve] add validation of expression values in cause exception

bfe9ef9

Hisoka-X previously approved these changes May 20, 2025

View reviewed changes

github-actions bot added the approved label May 20, 2025

github-actions bot removed approved reviewed labels May 22, 2025

misi added 12 commits May 27, 2025 10:31

[feature] throw sql transform exception,locate error expression

e88bc6e

[improve] throw TransformException

cba7ad7

[improve] add test case

276eea7

[improve] improve code

a0649d8

[improve] remove SqlTransformErrorCode\SqlTrasformException, reuse Tr…

fd1eee7

…ansforCommonError

[improve] 1.add where statement test case

259dcf3

2.remove throw TransformException 3.rename sqlTransformError to sqlExpressionError 4.modify EXPRESSION_EXECUTE_ERROR description 5.add where statement processing

[improve] add the exception into cause stack

f1c79a3

[improve] modify throw the sqlTransformException

8b59d9f

[improve] add validation of expression values in cause exception

41c3ac1

[improve] code format test

c70781a

[improve] code format test

0065e76

Merge remote-tracking branch 'origin/bugfix-9125' into bugfix-9125

16c5ee6

github-actions bot added document connectors-v2 e2e api labels May 27, 2025

Merge branch 'dev' into bugfix-9125

58bce88

github-actions bot removed document connectors-v2 e2e api labels May 27, 2025

[improve] the expression exception is wraped

a805e26

github-actions bot added the e2e label May 28, 2025

Hisoka-X approved these changes May 28, 2025

View reviewed changes

github-actions bot added approved reviewed labels May 28, 2025

hailin0 approved these changes May 28, 2025

View reviewed changes

hailin0 merged commit a67a2f7 into apache:dev May 28, 2025
5 checks passed


		import org.apache.seatunnel.common.exception.SeaTunnelErrorCode;

		public enum SqlTransformErrorCode implements SeaTunnelErrorCode {

	public static TransformException sqlTransformError(String transform, String expression) {
	public static TransformException sqlExpressionError(String transform, String expression) {

	"The expression '<expression>' of '<transform>' transform execute failed");
	"The expression '<expression>' of SQL transform execute failed");

	"ErrorCode:[TRANSFORM_COMMON-06], ErrorDescription:[The expression 'cast(`int` AS boolean)' of SQL transform execute failed]",
	"ErrorCode:[TRANSFORM_COMMON-06], ErrorDescription:[The expression 'CAST(`int` AS boolean)' of SQL transform execute failed]",

[Improve][Transform-V2] Improve sql transform exception to locate error expression #9227

[Improve][Transform-V2] Improve sql transform exception to locate error expression #9227

Uh oh!

Conversation

misi1987107 commented Apr 25, 2025

Purpose of this pull request

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Hisoka-X left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI May 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!